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Abstract 



> 

As additivity is a characteristic property of the classical information measure, Shan- 
non entropy, pseudo-additivity of the form x+ q y = x+y+(\—q)xy is a characteristic 
, property of Tsallis entropy. Renyi in [1] generalized Shannon entropy by means of 

Kolmogorov-Nagumo averages, by imposing additivity as a constraint. In this pa- 
per we show that there exists no generalization for Tsallis entropy, by means of 
Kolmogorov-Nagumo averages, which preserves the pseudo-additivity. 

^ (— I Key words: Kolmogorov-Nagumo averages, Renyi entropy, Tsallis entropy 

PACS: 65.40.Gr, 89.70.+C, 02.70.Rr 



The starting point of the theory of generalized measures of information is due 
to Alfred Renyi [1,2], now having an extensive literature. Renyi introduced 
generalized information measure, known as a-entropy or Renyi entropy, which 
is derived by replacing linear averaging in Shannon entropy with the general- 
ized averages, in particular Kolmogorov-Nagumo averages and by posing the 
additivity of the information measures. 

On the other hand, however, Tsallis in [3] proposed a non- logarithmic gener- 
alization of entropic measure, known as g-entropy or Tsallis entropy which is 
considered as a useful measure in describing thermostatistical properties of a 
certain class of physical systems that entail long-range interactions, long-term 
memories and multi-fractal systems. 



* corresponding author 

1 ambedkar@csa.iisc.ernet.in 

2 mnm@csa.iisc.ernet.in (Tel:+91-80-22932779) 

3 shalabh@csa.iisc.ernet.in 



Preprint submitted to Elsevier Science 



7 February 2008 



Tsallis and Renyi entropy measures are two possible different generalization 
of the Shannon entropy but are not generalizations of each other. 

To understand these generalizations, the so called Hartley information mea- 
sure [4] , of a single stochastic event plays a fundamental role. It can be inter- 
preted either as a measure of how unexpected the event is, or as a measure 
of the information yielded by the event. In a system with the finite configu- 
ration space x = {xk}k=i, Hartley information measure of a single event with 
probability pk is defined as 

H(p k )=\n-, k = l,...n. (1) 

Pk 



Hartley information measure satisfies: (1) H is nonnegative: H{p k ) > (2) H 
is additive: H(piPj) = H(pi) +H(pj) (3) H is normalized: H(j) = 1. These are 
both necessary and sufficient [5]. Hartley information measure can be viewed 
as a random variable and hence we use the notation H = (Hi, . . . H n ). 

Shannon entropy is defined as an average Hartley information: [6] 

n n 

S(p) = (H) = Y^PkHk = -Eft ln ft • (2) 

k=l k=l 



The characteristic property of Shannon entropy is additivity, i.e., for two in- 
dependent probability distributions p and r we have 

S(pr) = S(p) + S(r) , (3) 



where pr is the joint distribution of p and r. 

Renyi in [1,2] used a well known idea in mathematics that the linear mean, 
though most widely used, is not the only possible way of averaging, but one 
can define the mean with respect to an arbitrary function [7,8] to general- 
ize the Shannon entropy. In the general theory of means, a mean of x — 
(xi, x 2 , ■ ■ ■ , x n ) with respect to a probability distribution p — (pi,pi, . . . ,p n ) 
is defined as [7] 



' w 

J2Pk^ ( X k) 



.k=l 



(4) 



where ip is continuous and strictly monotonic (increasing or decreasing) in 
which case it has an inverse which satisfies the same conditions; ip is 
generally called the Kolmogorov-Nagumo function associated with the mean 
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(4) 4 . If, in particular, ip is linear, then (4) reduces to the expression of linear 
averaging, (x) = Y,k=iPk x k- The mean of form (4) is also referred as quasi- 
linear mean. 

In the definition of Shannon entropy (2), if the linear average of Hartley infor- 
mation is replaced with the generalized average of the form (4), the information 
corresponding to the probability distribution p with respect to KN-function ift 
will be 



-i 



n ( 1 

Y,Pki> hi — 



r 1 



J2 Pkip (H k ) 



Lk=l 



(5) 



where H = (Hi, . . . H n ) is the Hartley information measure associated with p. 
If we impose the constraint of additivity in S^, then -0 should satisfy [1] 

(x + C) i> = (x) if + C , (6) 



for any x — (xi, . . . , x n ) and a constant C. 

Renyi employed the above formalism to define an one-parameter family of 
measures of information (a-entropies) 



where the KN-function ip is chosen in (5) as vp(x) = e*- 1- "^, choice motivated 
by well known theorem in the theory of means (Theorem 89, [7]) that (6) 
can hold only for linear and exponential functions. Renyi entropy is an one- 
parameter generalization of Shannon entropy in the sense that, the limit a — > 1 
in (7) retrieves Shannon entropy. 

On the other hand, Tsallis entropy is given by [3] 

S q (p) = 1 ~^ Pl , (8) 



where q is called nonextensive index (q is positive in order to ensure the con- 
cavity of S q ). Tsallis entropy too, like Renyi entropy, is an one-parameter 

4 A. N. Kolmogorov [9] and M. Nagumo [10] were the first to investigate the charac- 
teristic properties of general means. They considered only the case of equal weights; 
the generalization to arbitrary weights and the characterization of means of form 
(4) are due to B. de Finetti [11], B. Jessen [12], T. Kitagawa [13], J. Aczel [8] and 
many others 
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generalization of Shannon entropy in the sense that q — > 1 in (8) retrieves 
Shannon entropy. The entropic index q characterizes the degree of nonexten- 
sivity reflected in the pseudo-additivity property 



S q (pr) = S q (p) + q S q {r) = S q (p) + S q (r) + (1 - q)S q (p)S q (r) , 



(9) 



where p and r are independent probability distributions. 

Though the derivation of Tsallis entropy, when it was proposed in 1988 is 
slightly different, one can understand this generalization using g-logarithm 
(see 11) function: where one would first generalize, logarithm in the Hartley 
information with ^-logarithm and define g-Hartley information measure H = 
(Hi,...,H n ) as 



H k = H(p k ) = ln q 



1 

Pk 



k — 1, . . . n , 



(10) 



where g-logarithm is defined as 
x 1 



\n q (x) 



_ i 



1-q 



:n) 



which satisfies pseudo-additivity \n q (xy) = ln g x+gln^ y and in the limit q — > 1 
we have ln^ — > \nx. Tsallis entropy (8) defined as the average of q-Hartley 
information i.e [14]: 



S g (p) 



H 



ln fi 



'12) 



Pk, 



Now a natural question arises whether one could generalize Tsallis entropy in 
the similar lines of derivation of Renyi entropy i.e., by replacing linear average 
in (12) by KN-averages under the pseudo-additivity. 

The class of information measures that represent the KN-average of g-Hartley 
information measure is written as 



S^(p) = (ln ? — ) = ifj 1 



w 



1 



.k% V Pk 



w 



Pk*P [H k 



.(13) 



By the pseudo-additivity constraint, ip should satisfy 
S^{pr) = S^(p) + q S^r) 



(14) 



or 
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r 1 



n n / 
i=1.7=l \ 5 . 



r 1 



i=l 



Pi. 



(15) 



where p and r are independent probability distributions and pr denotes the 
joint probability distribution of p and r. 

Equivalently, we need 



r 1 



n n 



i=i j=i 



r 1 



EftV* (3?) 



i=l 
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i=i 



(16) 



wherefP and H r represents the g-Hartley information of probability distribu- 
tions p and r respectively. 

Note that (16) must hold for arbitraryfinite discrete probability distributions 
Pi and Tj and for arbitrary numbers H k and H r k . If we choose H r k = J inde- 
pendently of j then (16) yields that 



r 1 



Lfc=l 



k=l 



+ q J 



(17) 



In general ip satisfies (17) only if satisfies 



(18) 



for any x — (x±, . . . , x n ), which can be rearranged as 
(x + C + (l- q)xC) i! = (x)^ + C + (1 - q)(x)^C 



(19) 



or 



((1 + (1 - q)C)x + C) . = (1 + (1 - q)C){x) . + C 



(20) 



Since q is independent of other quantities, if) should satisfy the equation of 
form (By B — (1 + (1 — q)C)) 



(Bx + C),=B(x),+C . 



(21) 
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Finally ip must satisfy 

<* + C)^ = (x) . + C 



(22) 



and 

(Bx)^ = B(x}^ , (23) 

for any x = (x±, . . . , x n ) and for any constants B and C, for to preserve 
the pseudo-additivity. 

From the generalized theory of means (22) is satisfied only when ip is linear 
or exponential, but the requirement (23) is satisfied only when if) is linear and 
it is not satisfied when if; is exponential. 

Hence if) is linear in which case (13) is nothing but Tsallis entropy. 

This establishes the nongeneralizability of Tsallis entropy by means of KN- 
averages under pseudo-additivity. 
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